Segmentation of Folk Songs with a Probabilistic Model

Structure is an important aspect of music. Musical structure can be recognized in different musical modalities such as rhythm, melody, harmony or lyrics and plays a crucial role in our appreciation of music. In recent years many researchers have addressed the problem of music segmentation, mainly for popular and classical music. Some of the more recent approaches are Mauch et al. (2009), Foote (2000), Serr`a et al. (2012) and McFee & Ellis (2014). Last three are included in the music structure analysis framework MSAF Nieto & Bello (2015). None of the mentioned approaches however, addresses the specifics of folk music. While commercial music is performed by professional performers and recorded with professional equipment in suitable recording conditions, this is usually not true for folk music field recordings, which are recorded in everyday environments and contain music performed by amateur performers. Thus, recordings may contain high levels of background noise, equipment induced noise (e.g. hum) and reverb, as well as performer mistakes such as inaccurate pitches, false starts, forgotten melody/lyrics or pitch drift throughout the performance. One of the most recent approaches which addressed folk music specifics was presented by M¨uller et al. (2013). The approach was designed for solo singing and was evaluated on a collection of Dutch folk music by Muller et al. (2010). In our paper, we present a novel folk music segmentation method, which also addresses folk music specifics and is designed to work well with a variety of ensemble types (solo, choir, instrumental and mixtures)

Finding the most representative part of vocal folksongs with transcription and segmentation

The goal of musical segmentation is to develop algorithms that will find similar patterns in audio signal according to desired aspect (melody, rhythm, timbre) and to define the boundaries between the repetitions. The goal of musical transcription is to develop algorithms that will extract pitches from the audio signal in every time frame either for monophonic or polyphonic music. Music segmentation and transcription represent two very important parts of music information retrieval research field. The results can be used in many real-life applications: with music segmentation we can define musical structure, melodic repetitions in music or we can use it in search for most representative part; transcription results can be used in automatic generation of scores, as a support in manual transcription process or in search of similar melodies in musical collections. In the presented dissertation we are addressing specific problems of musical segmentation and transcription of audio recordings: segmentation and transcription of folk music audio recordings. Currently developed methods fail on folk music due to it's specifics, such as bad recording conditions and amateur performers, which are the reason for high level of noise in recordings, inaccurate singing, pitch drifting throughout the song etc. In introduction section we give the motivation for conducting the research and define the problems and goals of the thesis in the detail. The first part of the dissertation presents the research from field of music segmentation, where we present a folk music segmentation method, that outperforms current state-of-the-art methods on a collection of folk music. The presented segmentation method bases on a probabilistic model for finding melodically repeating parts in recording and defining their beginnings. The method was evaluated on a folk music collection of different types: solo singing, two- and three-voiced singing, choir songs, instrumental songs and mixed assembles. The developed method was also evaluated according to robustness aspect, where resistance to different degradations was tested and evaluated. The second part of the dissertation addresses musical transcription, where we present a folk music transcription method. The method uses the segmentation results to find a representative part of a song and transcribes it with use of all the repetitions within the song. The method takes multiple fundamental frequencies estimations calculated with an existing method and song segmentation. With use of segmentation results the method aligns the multiple fundamental frequencies estimations in temporal and frequency domain, removes local inaccuracies and joins the transcriptions of all repeating parts. In next stage the method calculates notes using two-level probabilistic model based on explicit duration Hidden Markov models, used to model notes, rests and note transitions. The presented method was evaluated on collection of polyphonic folk music, where it returns better results of current state-of-the-art music transcription methods. In the conclusions we highlight the scientific contributions of the thesis and give the directions for possible future improvements and extensions of the method

Dr. KID: Direct Remeshing and K-set Isometric Decomposition for Scalable Physicalization of Organic Shapes

Dr. KID is an algorithm that uses isometric decomposition for the physicalization of potato-shaped organic models in a puzzle fashion. The algorithm begins with creating a simple, regular triangular surface mesh of organic shapes, followed by iterative k-means clustering and remeshing. For clustering, we need similarity between triangles (segments) which is defined as a distance function. The distance function maps each triangle's shape to a single point in the virtual 3D space. Thus, the distance between the triangles indicates their degree of dissimilarity. K-means clustering uses this distance and sorts of segments into k classes. After this, remeshing is applied to minimize the distance between triangles within the same cluster by making their shapes identical. Clustering and remeshing are repeated until the distance between triangles in the same cluster reaches an acceptable threshold. We adopt a curvature-aware strategy to determine the surface thickness and finalize puzzle pieces for 3D printing. Identical hinges and holes are created for assembling the puzzle components. For smoother outcomes, we use triangle subdivision along with curvature-aware clustering, generating curved triangular patches for 3D printing. Our algorithm was evaluated using various models, and the 3D-printed results were analyzed. Findings indicate that our algorithm performs reliably on target organic shapes with minimal loss of input geometry

Probabilistic Segmentation of Folk Music Recordings

The paper presents a novel method for automatic segmentation of folk music field recordings. The method is based on a distance measure that uses dynamic time warping to cope with tempo variations and a dynamic programming approach to handle pitch drifting for finding similarities and estimating the length of repeating segment. A probabilistic framework based on HMM is used to find segment boundaries, searching for optimal match between the expected segment length, between-segment similarities, and likely locations of segment beginnings. Evaluation of several current state-of-the-art approaches for segmentation of commercial music is presented and their weaknesses when dealing with folk music are exposed, such as intolerance to pitch drift and variable tempo. The proposed method is evaluated and its performance analyzed on a collection of 206 folk songs of different ensemble types: solo, two- and three-voiced, choir, instrumental, and instrumental with singing. It outperforms current commercial music segmentation methods for noninstrumental music and is on a par with the best for instrumental recordings. The method is also comparable to a more specialized method for segmentation of solo singing folk music recordings

Edoo: An Online Match-making Portal for Educational Content Production

Although masses of electronic learning materials are being regularly created for e-learning purposes, there is still difficult for a teacher to find a suitable material for a particular teaching situation. Instead of adapting lessons to suit the available learning content, the teachers should actively adjust the learning content itself to make it suit their needs for use in the class. Despite having good ideas, not all teachers are capable of creating an attractive learning content, or even just customising it due to the lack of programming knowledge and inadequate ICT-usage skills. Our goal is to bring together two distinct communities, teachers and programmers, to work together, share ideas, and brainstorm, with the common goal to benefit from this mashing in providing useful materials for enhancing learning experience. The portal is aimed as a meeting place for teachers with innovative ideas for new e-content and technology buffs wishing to contribute their knowledge to common good

Usability evaluation of input devices for navigation and interaction in 3D visualisation

We present an assessment study of user experience and usability of different kinds of input devices for view manipulation in a 3D data visualisation application. Three input devices were compared: a computer mouse, a 3D mouse with six degrees of freedom, and the Leap Motion Controller - a device for touchless interaction. Assessment of these devices was conducted using the System Usability Scale (SUS) methodology, with addition of application specific questions. To gain further insight into users' behaviour, the users' performance and feedback on the given tasks was recorded and analysed. The best results were achieved by using the 3D mouse (SUS score 88.7), followed by the regular mouse (SUS score 72.4). The Leap Motion Controller (SUS score 56.5) was the least preferred mode of interaction, nevertheless it was described as natural and intuitive, showing great potential

Finding Nano-\"Otzi: Semi-Supervised Volume Visualization for Cryo-Electron Tomography

Cryo-Electron Tomography (cryo-ET) is a new 3D imaging technique with unprecedented potential for resolving submicron structural detail. Existing volume visualization methods, however, cannot cope with its very low signal-to-noise ratio. In order to design more powerful transfer functions, we propose to leverage soft segmentation as an explicit component of visualization for noisy volumes. Our technical realization is based on semi-supervised learning where we combine the advantages of two segmentation algorithms. A first weak segmentation algorithm provides good results for propagating sparse user provided labels to other voxels in the same volume. This weak segmentation algorithm is used to generate dense pseudo labels. A second powerful deep-learning based segmentation algorithm can learn from these pseudo labels to generalize the segmentation to other unseen volumes, a task that the weak segmentation algorithm fails at completely. The proposed volume visualization uses the deep-learning based segmentation as a component for segmentation-aware transfer function design. Appropriate ramp parameters can be suggested automatically through histogram analysis. Finally, our visualization uses gradient-free ambient occlusion shading to further suppress visual presence of noise, and to give structural detail desired prominence. The cryo-ET data studied throughout our technical experiments is based on the highest-quality tilted series of intact SARS-CoV-2 virions. Our technique shows the high impact in target sciences for visual data analysis of very noisy volumes that cannot be visualized with existing techniques